feat: free resources less often; fix: handle non-standard inpainting payloads #199
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR has two major components:
Removing calls to garbage collect ram/vram
This is an experiment to try and give a better chance at having loaded models remain in VRAM/RAM. The
cleanup()
andgarbage_collect()
functions date back to a time when hordelib was doing much more in reaching into ComfyUI and changing behavior surrounding memory allocation. This change may allow ComfyUI internally to handle better when it feels memory should or should not be free.As for the worker use-case, the worker code already manually calls equivalents of these functions at the appropriate times, and as such these calls should be entirely redundant for non-worker use-cases, and may serve to conflict somewhat with worker use-cases.
Fixes to inpainting model handling
Certain payloads for inpainting models (especially ones that lack a
source_image
and/orsource_mask
) cause ComfyUI to throw an exception. In the case of the worker, this causes the entire job to fault and potentially leads to workers getting placed into maintenance mode. This PR introduces the behavior of defaulting source_image/source_mask if they are not present.There are warning/error level logging messages to this effect, so in the off chance hordelib is being used standalone, those users will be clued into this potentially surprising behavior. Horde users will receive
GEN_METADATA
objects as appropriate.